Skip to content

feat: AI transcript summaries with Claude - #10

Merged
Adeolu05 merged 4 commits into
mainfrom
feat/ai-summaries
Oct 13, 2026
Merged

Adeolu05 merged 4 commits into
mainfrom
feat/ai-summaries

Conversation

@Adeolu05

Copy link
Copy Markdown
Owner

Builds on #9 (the language picker and the earlier fixes). Until #9 merges, this diff also shows #9's commits. Merging in either order is safe.

Summary

Optional AI summary of any extracted transcript: a TL;DR, 3–7 key points, and timestamped chapters, written in the transcript's language.

  • Web: a "Summarize with AI" card under the preview. YouTube chapter timestamps link to that moment in the video.
  • Telegram: an "AI summary" button on the ready message.
  • API: POST /api/v1/summarize {file_id}. /config reports summaries_enabled.

Model & request

  • claude-opus-5-5 (configurable via SUMMARY_MODEL) at effort: low (SUMMARY_EFFORT). Summarizing is a simple task, so low effort keeps thinking tokens and cost down.
  • The output is constrained by a JSON schema (output_config.format), so there's no fragile parsing.
  • Server-side refusal fallback is on (fallbacks: "default", beta server-side-fallback-2026-07-01). If a safety classifier declines a transcript, Anthropic retries it on its recommended fallback model instead of failing. stop_reason is checked (refusal, max_tokens) before any content is read.
  • The transcript is sent as untrusted data inside <transcript> tags, and the system prompt tells the model to ignore instructions inside it (captions are arbitrary third-party text).
  • Caption cues are merged into ~20s [m:ss] lines, which cuts tokens while keeping chapter timestamps accurate. Transcripts over SUMMARY_MAX_INPUT_CHARS (200K chars ≈ 50K tokens) are refused with a clear message, never silently truncated.

Cost controls

  • Summaries are cached per (provider, video, language) alongside the caption cache (same TTL and pruning). A repeat request, from anyone, costs nothing.
  • A new summarize quota bucket (default 3 per window, per IP or Telegram user) is charged only on a cache miss.
  • Every Claude call logs summary_generated with input_tokens / output_tokens / model / latency, so you can measure real cost per summary before deciding on pricing.
  • Off by default: without ANTHROPIC_API_KEY the button is hidden and the endpoint returns 503.

Privacy: the privacy page and the bot's /privacy text now say that clicking "summarize" sends the caption text to Anthropic.

Test plan

  • 111 unit tests pass (12 new, with a fake Anthropic client): request shape (model, fallbacks, beta, effort, schema, untrusted-data framing), output cleanup, per-language caching, refusal → 422 with nothing cached, max_tokens, too-long refused without calling Claude, config flag, 503 when disabled, cache hit skips both Claude and the quota, quota on a miss, 404 for an unknown file
  • eslint + tsc --noEmit clean
  • Live call not yet run: no Anthropic credentials on the dev machine. Set ANTHROPIC_API_KEY and summarize one video, then check the summary_generated log line for token counts.
  • Telegram: "AI summary" button end to end

🤖 Generated with Claude Code

Adeolu05 and others added 4 commits September 28, 2026 18:44
- Fetch the YouTube title via oEmbed, concurrently with captions, instead of
  a sequential watch-page scrape (up to 10s) before the caption request.
  Duration now comes from the guardrails' last-cue estimate.
- Carry a monotonic deadline from the API/bot into the fetch: every YouTube
  request is capped at the time left, IP-block retries never sleep past it,
  and Vimeo requests are bounded too. Timed-out extracts stop calling
  upstream instead of running on in the worker thread.
- New UPSTREAM_BLOCKED error code (503) for IpBlocked/RequestBlocked so the
  metrics dashboard separates proxy failures from videos without captions;
  YouTube timeouts map to UPSTREAM_TIMEOUT (504), network errors to a clear
  message instead of a 500.
- CI: lint, type-check, and build the web app.
- Web: remove seven unused components from an earlier homepage and fix the
  remaining lint errors (useHydrated replaces setState-in-effect guards).

Co-Authored-By: Claude Opus 5.5 <[email protected]>
… logs

- REDIS_URL (optional): extract/convert/events quotas are counted once
  across all Gunicorn workers and the Telegram bot instead of per process;
  Telegram format/timestamp preferences survive restarts (90-day TTL).
  Unset or unreachable Redis falls back to in-memory with a throttled warning.
- Docker Compose runs a small Redis; render.yaml takes REDIS_URL and
  generates LOG_HASH_SECRET.
- hash_identifier is now HMAC-keyed (plain SHA-256 of an IPv4 or Telegram ID
  is reversible by enumeration); rate-limit block logs no longer contain the
  raw client IP.
- Failure logs record the public video ID + host instead of a partially
  stripped URL (which lost the ID for watch?v= links and kept share tokens
  in youtu.be paths).
- Privacy page / bot copy: document IP-based limits, hashed log IDs, and
  Telegram preferences; caption cache lasts days (72h default), not hours.
- CI: Redis service container for real-server integration tests.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
- Web picker + Telegram /language: choose the transcript language; YouTube
  uses a native track or auto-translates, Vimeo picks the matching uploaded
  track. Responses say what came back (translated / fallback) and the UI
  explains it. Language list served from /api/v1/config.
- Correct Chinese handling: zh-Hans and zh-Hant no longer treated as equal.
- Cache key includes the language.
- Translation failures from blocks/timeouts now surface as errors instead of
  silently returning (and caching) original-language captions.
- PDFs: fonts chosen from the actual text. Embedded DejaVu Sans for
  Latin-extended/Cyrillic/Greek, built-in CID fonts for Chinese/Japanese,
  embedded Nanum Gothic for Korean; Arabic/Hebrew/Indic/Thai get a clear
  "PDF not available" (DOCX/TXT still work) instead of broken glyphs.
  Previously every PDF used Helvetica, so e.g. Polish or Russian text came
  out as empty boxes.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
- POST /api/v1/summarize and a Telegram "AI summary" button, backed by one
  summary service calling Claude Opus 5.5 at low effort with a JSON-schema
  output (tldr, key_points, chapters with start_seconds), written in the
  transcript's language.
- Server-side refusal fallback (fallbacks: "default") enabled; refusal and
  max_tokens stop reasons handled before reading content.
- Transcript passed as untrusted data inside <transcript>, compacted into
  ~20s timestamped lines; over-long transcripts refused, not truncated.
- Summaries cached per (provider, video, language) with the caption cache;
  the new "summarize" quota (default 3/window) is only charged on a cache miss.
- Feature hidden unless ANTHROPIC_API_KEY is set (/config summaries_enabled).
- Token usage logged per call (summary_generated) to measure cost.
- Web: summary card with chapter links into the YouTube video; privacy page
  and bot privacy text disclose that captions go to Anthropic on request.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@vercel

vercel Bot commented Sep 28, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
transcriptflow Ready Ready Preview Sep 28, 2026 10:26pm UTC

@Adeolu05
Adeolu05 merged commit bc159f4 into main Oct 13, 2026
4 checks passed

This branch was successfully deployed

1 active deployment
Preview — 1ff6b19c Deployed Sep 28, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant